Papers with causal model
From Generating Answers to Building Explanations: Integrating Multi-Round RAG and Causal Modeling for Scientific QA (2025.naacl-industry)
Copied to clipboard
Victor Barres, Clifton James McFate, Aditya Kalyanpur, Kailash Karthik Saravanakumar, Lori Moon, Natnael Seifu, Abraham Bautista-Castillo
| Challenge: | Application of Large Language Models to complex causal question answering can be stymied by their opacity and propensity for hallucination. |
| Approach: | They propose a causal QA approach that combines iterative RAG with a formal model of causation. |
| Outcome: | The proposed approach is implemented into a Collaborative Research Assistant (Cora) and evaluated in the life sciences domain. |
Understanding Demonstration-based Learning from a Causal Perspective (2023.acl-short)
Copied to clipboard
| Challenge: | Demonstration-based learning has shown impressive performance in exploiting pretrained language models under few-shot learning settings. |
| Approach: | They propose to construct a Structural Causal Model to understand demonstration-based learning from causal perspectives and interpret random demonstrations as interventions on the demonstration variable within the causal model. |
| Outcome: | The proposed model outperforms hand-crafted demonstrations on public sequence labeling benchmarks. |
Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddings (2023.eacl-main)
Copied to clipboard
| Challenge: | Past work on sentence embedding models faces issues determining the causal impact of implicit syntax representations. |
| Approach: | They construct a neural module net based on a transformer model and train it end-to-end to approximate the sentence’s embedding. |
| Outcome: | The proposed model captures whether syntax is a strong model of its compositional ability. |
Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens (2022.emnlp-main)
Copied to clipboard
| Challenge: | 'spurious correlations' have been used in NLP to informally denote any undesirable feature-label correlations. |
| Approach: | They formalize this distinction using a causal model and probabilities of necessity and sufficiency, which delineates causal relations between a feature and a label. |
| Outcome: | The proposed model is invariant to the feature, but not sufficient for prediction. |
Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Language models operating on subword units are challenging for character-level manipulations, authors say . authors develop a framework to learn robust character representations inside subword-based models . |
| Approach: | They propose a causal intervention framework to learn robust character representations inside subword-based language models. |
| Outcome: | The proposed model outperforms character-level models on more complex tasks . it improves robustness on unseen token sequences and leads to human-interpretable representations of characters. |